Receiving system and method of processing data

ABSTRACT

A receiving system that can receive and process 3D images and a data processing method of the same are disclosed. The receiving system includes an image receiving unit and a display unit. The image receiving unit receives a 3-dimensions (3D) image and system information including additional information of the 3D image (i.e., additional 3D image information), generates 3D signaling information based upon the additional 3D image information included in the system information, and transmits the generated 3D signaling information along with the 3D image through a digital interface. And, the display unit receives the 3D signaling information along with the 3D image through the digital interface, formats the 3D image based upon the receiving 3D signaling information, and displays the formatted 3D image.

This application is a continuation of application Ser. No. 12/588,302filed on Oct. 9, 2009, which claims the benefit of U.S. ProvisionalApplication No. 61/104,274, filed on Oct. 10, 2008, which is herebyincorporated by reference as if fully set forth herein.

Application Ser. No.12/588,302 also claims the benefit of U.S.Provisional Application No. 61/240,657, filed on Sep. 9, 2009, and U.S.Provisional Application No. 61/173,196, filed on Apr. 27, 2009, each ofwhich is hereby incorporated by reference as if fully set forth herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and device for processing animage signal and, more particularly, to a receiving system for receivingand processing a 3-dimensional (3D) image signal and a method ofprocessing data.

2. Discussion of the Related Art Generally, a 3-dimensional (3D) image(or stereoscopic image) is based upon the principle of stereoscopicvision of both human eyes. A parallax between both eyes, in other words,a binocular parallax caused by the two eyes of an individual beingspaced apart at a distance of approximately 65 millimeters (mm) isviewed as the main factor that enables the individual to view objects3-dimensionally. When each of the left eye and the right eyerespectively views a 2-dimensional (or flat) image, the brain combinesthe pair of differently viewed images, thereby realizing the depth andactual form of the original 3D image.

Such 3D image display may be broadly divided into a stereoscopic method,a volumetric method, and a holographic method. Furthermore, the methodof displaying 3D images may be broadly divided into a method of wearingspecial glasses and a method of not wearing any special glasses.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a receiving system anda method of processing data that substantially obviate one or moreproblems due to limitations and disadvantages of the related art.

An object of the present invention is to provide a receiving system anda method for processing data that can identify the reception of a 3Dimage and process the received 3D image.

Another object of the present invention is to provide a receiving systemand a method for processing data that can supply additional informationassociated to 3D images, which are included in and received via systeminformation, to a display device connected to a digital interface.

Additional advantages, objects, and features of the invention will beset forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of theinvention. The objectives and other advantages of the invention may berealized and attained by the structure particularly pointed out in thewritten description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, areceiving system includes an image receiving unit and a display unit.Herein, the image receiving unit may also be referred to as a decodingdevice (or an HDMI source), and the display unit may also be referred toas a display device (or an HDMI sink). More specifically, the imagereceiving unit receives a 3-dimensions (3D) image and system informationincluding additional information of the 3D image (i.e., additional 3Dimage information), generates 3D signaling information based upon theadditional 3D image information included in the system information, andtransmits the generated 3D signaling information along with the 3D imagethrough a digital interface. And, the display unit receives the 3Dsignaling information along with the 3D image through the digitalinterface, formats the 3D image based upon the receiving 3D signalinginformation, and displays the formatted 3D image.

Herein, the additional 3D image information may be included in a programmap table (PMT) of the system information in a descriptor format,thereby being received. The additional 3D image information may includea field indicating whether the 3D image is being received, a fieldindicating a transmission format of the 3D image, a field indicatingwhether an uppermost pixel of a left-end portion within the receivedimage frame belongs to a left image or to a right image, a fieldindicating whether at least one of the left image and the right imagehas been inversely scanned and encoded, a field indicating which one ofthe left image and the right image has been inversely scanned, and afield indicating whether at least one of the left image and the rightimage has been sampled by using a filter.

The HDMI source may insert and transmit the 3D signaling informationgenerated from the additional 3D image information in an AVI InfoFramepacket. Furthermore, the AVI InfoFrame packet may comprises of a headerand a contents region, and one or more fields may be assigned to atleast one byte of the contents region, thereby recording the 3Dsignaling information. Also, at least one byte of the contents region inthe AVI InfoFrame packet may include a field indicating whether the 3Dimage is being received, a field indicating a transmission format of the3D image, a field indicating whether an uppermost pixel of a left-endportion within the received image frame belongs to a left image or to aright image, a field indicating whether at least one of the left imageand the right image has been inversely scanned, a field indicating whichone of the left image and the right image has been inversely scanned,and a field indicating whether at least one of the left image and theright image has been sampled by using a filter.

In another aspect of the present invention, a data processing method ina receiving system includes receiving a 3-dimensions (3D) image andsystem information including additional information of the 3D image(i.e., additional 3D image information), generating 3D signalinginformation based upon the additional 3D image information included inthe system information, and transmitting the generated 3D signalinginformation along with the 3D image through a digital interface, andreceiving the 3D signaling information along with the 3D image throughthe digital interface, formatting the 3D image based upon the receiving3D signaling information, and displaying the formatted 3D image.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates examples of a single video stream format amongtransmission formats for 3D images according to the present invention;

FIG. 2 illustrates examples of a multiple video stream format amongtransmission formats for 3D images according to the present invention;

FIG. 3 illustrates a PMT syntax structure, wherein identificationinformation that can identify the reception of a 3D image is included ina descriptor format, according to an embodiment of the presentinvention;

FIG. 4 illustrates a syntax structure of a stereoscopic video formatdescriptor according to the embodiment of the present invention;

FIG. 5 illustrates a schematic view of a 3D image system according tothe embodiment of the present invention;

FIG. 6 illustrates a block view showing an exemplary structure of adecoding device and a display device being interconnected by an HDMIcable within the receiving system according to the present invention;

FIG. 7 illustrates a table showing example of diverse packet types beingused in an HDMI standard according to the present invention;

FIG. 8 illustrates an exemplary header structure of an AVI InfoFramepacket according to the embodiment of the present invention;

FIG. 9 illustrates a content structure of a general AVI InfoFrame packetaccording to the embodiment of the present invention;

FIG. 10 illustrates a content structure of an AVI InfoFrame packetaccording to the embodiment of the present invention;

(a) to (e) of FIG. 11 respectively illustrate a table indicating themeaning of values assigned to each field within the 15^(th) byte of theAVI InfoFrame packet according to the embodiment of the presentinvention;

FIG. 12 illustrates a flow chart showing process steps for generatingand transmitting 3D signaling information according to the embodiment ofthe present invention; and

FIG. 13 illustrates a flow chart showing process steps for processingvideo data by referring to the 3D signaling information and displayingthe processed video data as a 3D image according to the embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts. In addition,although the terms used in the present invention are selected fromgenerally known and used terms, some of the terms mentioned in thedescription of the present invention have been selected by the applicantat his or her discretion, the detailed meanings of which are describedin relevant parts of the description herein. Furthermore, it is requiredthat the present invention is understood, not simply by the actual termsused but by the meaning of each term lying within.

The present invention relates to a receiving system that can recognizethe reception of a 3D image and that can process the received 3D image.According to the embodiment of the present invention, a transmittingsystem of the present invention includes supplemental (or additional)information on the 3D image in system information and transmits thesupplemental information.

The present invention also relates to using the supplemental informationon the 3D image included in the system information and received by thereceiving system, so as to decode the 3D image.

The present invention also relates to generating 3D signalinginformation from additional information of the 3D image (or additional3D image information), which is included in the system information andreceived, thereby providing the generated 3D signaling information to adisplay device connected to a digital interface.

Finally, the present invention relates to having the display deviceprocess 3D images based upon the 3D signaling information, which isprovided through the digital interface, thereby displaying the processed3D images.

Herein, 3D images may include stereo (or stereoscopic) images, whichtake into consideration two different perspectives (or viewpoints), andmulti-view images, which take into consideration three differentperspectives.

A stereo image refers to a pair of left and right images acquired byphotographing the same subject with a left-side camera and a right-sidecamera, wherein both cameras are spaced apart from one another at apredetermined distance. Furthermore, a multi-view image refers to a setof at least 3 images acquired by photographing the same subject with atleast 3 different cameras either spaced apart from one another atpredetermined distances or placed at different angles. Although thestereo (or stereoscopic) image has been described as an embodiment ofthe present invention, it is apparent that multi-view images may also beapplied to the present invention.

The transmission formats of stereo images include a single video streamformat and a multi-video stream format.

Herein, the single video stream format includes a side-by-side formatshown in (a) of FIG. 1, a top/down format shown in (b) of FIG. 1, aninterlaced format shown in (c) of FIG. 1, a frame sequential formatshown in (d) of FIG. 1, a checker board format shown in (e) of FIG. 1,and an anaglyph format shown in (f) of FIG. 1.

Also, the multi-video stream format includes a full left/right formatshown in (a) of FIG. 2, a full left/half right format shown in (b) ofFIG. 2, and a 2D video/depth format shown in (c) of FIG. 2.

For example, the side-by-side format shown in (a) of FIG. 1 correspondsto a case where a left image and a right image are ½ sub-sampled in ahorizontal direction. Herein, the sampled left image is positioned onthe left side, and the sampled right image is positioned on the rightside, thereby creating a single stereo image. The top/down format shownin (b) of FIG. 1 corresponds to a case where a left image and a rightimage are ½ sub-sampled in a vertical direction. Herein, the sampledleft image is positioned on the upper side, and the sampled right imageis positioned on the lower side, thereby creating a single stereo image.The interlaced format shown in (c) of FIG. 1 corresponds to a case wherea left image and a right image are ½ sub-sampled in a horizontaldirection. Herein, pixels of the sampled left image and pixels of thesampled right image are alternated line by line, thereby creating asingle stereo image. Alternatively, a left image and a right image are ½sub-sampled in a horizontal direction, and pixels of the sampled leftimage and pixels of the sampled right image are alternated pixel bypixel (i.e., in single pixel units), thereby creating a single stereoimage. The checker board format shown in (e) of FIG. 1 corresponds to acase where a left image and a right image are ½ sub-sampled in bothhorizontal and vertical directions. Herein, pixels of the sampled leftimage and pixels of the sampled right image are alternated in singlepixel units, thereby creating a single stereo image.

Furthermore, the full left/right format shown in (a) of FIG. 2corresponds to a case where a left image and a right image aresequentially transmitted. The full left/half right format shown in (b)of FIG. 2 corresponds to a case where the left image remains in itsoriginal state, and where the right image is ½ sub-sampled either in ahorizontal direction or in a vertical direction. Finally, the 2Dvideo/depth format shown in (c) of FIG. 2 corresponds to a case whereone of the left image and the right image is transmitted, and wheredepth information for creating another image is also transmitted at thesame time.

Also, in case of the receiving system corresponds to a system that canprocess 3D images, the receiving system should be able to recognize thereception of a 3D image. Furthermore, since 3D images may be transmittedin diverse transmission formats, the receiving system should also beinformed of the receiving format of the 3D image so as to be able torecover the 3D image to its initial (or original image). According to anembodiment of the present invention, in order to do so, the transmittingsystem of the present invention should include additional information onthe 3D image (or additional 3D image information) in the systeminformation, when transmitting the system information. According to theembodiment of the present invention, the receiving system of the presentinvention uses the additional 3D image information included in thereceived system information, thereby decoding the received 3D image anddisplaying the decoded 3D image.

In some cases, the system information may also be referred to as serviceinformation. Herein, the system information may include channelinformation, program information, event information, and so on.

According to the embodiment of the present invention, a program specificinformation/program and system information protocol (PSI/PSIP) isadopted as the system information. However, the present invention willnot be limited only to this example. In other words, any protocol thattransmits system information in a table format may be applied in thepresent invention regardless of the name of the corresponding protocol.

The PSI table is an MPEG-2 system standard defined for dividing (orcategorizing) channels and programs. The PSIP table is an advancedtelevision systems committee (ATSC) standard that can enable thedivision (or identification or categorization) of the channels and theprograms. According to an embodiment of the present invention, the PSItable may include a program association table (PAT), a conditionalaccess table (CAT), a program map table (PMT), and a network informationtable (NIT).

Herein, the PAT corresponds to special information that is transmittedby a data packet having a PID of ‘0’. The PAT transmits PID informationof the corresponding PMT and PID information of the corresponding NITfor each program. The CAT transmits information on a paid broadcastingsystem used by a transmitting system. The PMT transmits PID informationof a transport stream (TS) packet, in which program identificationnumbers and individual bit sequences of video and audio data configuringthe corresponding program are transmitted, and also transmits the PIDinformation in which PCR is transmitted. The NIT transmits informationof the actual transmission network. For example, by parsing a PAT tablehaving the PID of ‘0’, a program number and a PID of the PMT may befound (or acquired). Then, by parsing the PMT acquired from the PAT, thecorrelation between the elements configuring the corresponding programmay also be acquired (or found).

According to an embodiment of the present invention, the PSIP table mayinclude a virtual channel table (VCT), a system time table (STT), arating region table (RRT), an extended text table (ETT), a directchannel change table (DCCT), an event information table (EIT), and amaster guide table (MGT).

The VCT transmits information on virtual channels, such as channelinformation for selecting channels and information such as packetidentification (PID) numbers for receiving the audio and/or video data.More specifically, when the VCT is parsed, the PID of the audio/videodata of the broadcast program may be known. Herein, the correspondingaudio/video data are transmitted within the channel along with thechannel name and channel number. The STT transmits information on thecurrent data and timing information. The RRT transmits information onregion and consultation organs for program ratings. The ETT transmitsadditional description of a specific channel and broadcast program. TheEIT transmits information on virtual channel events (e.g., programtitle, program start time, etc.). The DCCT/DCCSCT transmits informationassociated with automatic (or direct) channel change. And, the MGTtransmits the versions and PID information of the above-mentioned tablesincluded in the PSIP.

According to an embodiment of the present invention, the additional 3Dimage information may be included in the system information, so as to bereceived, in at least one or more descriptor format or field format.

Also, according to an embodiment of the present invention, theadditional 3D image information may be included in a PMT within thesystem information in a descriptor format, so as to be received.

Furthermore, according to an embodiment of the present invention, theadditional 3D image information may also be included in a VCT within thesystem information in a descriptor format, so as to be received.

The additional 3D image information may include at least one or moreinformation associated to the 3D image. The additional 3D imageinformation may also include information indicating the transmissionformat of the 3D image, information indicating whether the receivedimage corresponds to a 2D image or a 3D image, information indicatingwhether the uppermost pixel of the left-end portion within the receivedimage frame belongs to a left image or to a right image, informationindicating whether at least one of the left image and the right imagehas been inversely scanned and encoded, information indicating which oneof the left image and the right image has been inversely scanned, andinformation indicating whether at least one of the left image and theright image has been sampled by using a filter.

FIG. 3 illustrates a PMT syntax structure including additional 3D imageinformation in a descriptor format according to an embodiment of thepresent invention.

Referring to FIG. 3, a table_id field corresponds to a table identifier.Herein, an identifier that identifies the PMT may be set as the table_idfield.

A section_syntax_indicator field corresponds to an indicator defining asection format of the PMT.

A section_length field indicates the section length of the PMT.

A program_number field corresponds to information matching with the PAT.Herein, the program_number field indicates the number of thecorresponding program.

A version number field indicates a version number of the PMT.

A current_next_indicator field corresponds to an indicator indicatingwhether the current table section is applicable or not.

A section_number field indicates the section number of the current PMTsection, when the PMT is divided into at least one or more sections,thereby being transmitted.

A last_section_number field indicates the last section number of thecorresponding PMT.

A PCR_PID field indicates the PID of a packet that delivers a programclock reference (PCR) of the current program.

A program_info_length field indicates length information of a descriptorimmediately following the program_info_length field in number of bytes.More specifically, the program_info_length field indicates the length ofeach descriptor included in the first loop.

A stream_type field indicates a type of element stream and encodinginformation included in a packet having the PID value marked in anelementary_PID field that follows. For example, according to anembodiment of the present invention, if the corresponding stream is anMPEG-2-encoded video stream, the stream_type field value may be markedto have a value of ‘0x02’.

The elementary_PID field indicates an identifier of the element stream,i.e., the PID value of a packet including the corresponding elementstream. For example, if the stream_type field value is equal to ‘0x02’,the elementary_PID field indicates a PID of an MPEG-2-encoded video ES.

An ES_info_length field indicates the length information of a descriptorimmediately following the ES_info_length field in number of bytes. Morespecifically, the ES_info_length field indicates the length of eachdescriptor included in the second loop.

According to the present invention, descriptors of a program level areincluded in the descriptor( ) region within the first loop of the PMT,and descriptors of a stream level are included in the descriptor( )region within the second loop of the PMT. More specifically, thedescriptors included in the first loop correspond to descriptors thatare individually applied to each program, and the descriptors includedin the second loop correspond to descriptors that are individuallyapplied to each ES.

According to an embodiment of the present invention, if a programcorresponding to the program_number field value of the PMT is a 3Dimage, i.e., a 3D content, additional 3D image information is includedin a descriptor( ) region of the second loop within the PMT in adescriptor format. In the description of the present invention, theabove-described descriptor will be referred to as a stereoscopic videoformat descriptor stereoscopic_video_format_descriptor( ). For example,when the elementary_PID field value indicates the PID of the video ES,the stereoscopic video format descriptorstereoscopic_video_format_descriptor( ) is included after theES_Info_length field. The stereoscopic video format descriptorstereoscopic_video_format_descriptor( ) may also be included in thefirst loop of the PMT.

More specifically, in the receiving system, if the stereoscopic videoformat descriptor is included in the second loop of the PMT, therebybeing received, the stereoscopic video format descriptor is parsed so asto obtain the additional 3D image information.

FIG. 4 illustrates a syntax structure of the stereoscopic video formatdescriptor stereoscopic_video_format_descriptor( ) according to anembodiment of the present invention.

Herein, a descriptor_tag field is assigned with 8 bits and indicatesthat the corresponding descriptor isstereoscopic_video_format_descriptor( ).

A descriptor_length field is an 8-bit field, which indicates the bytesize (or length) starting from the end of the descriptor_length field tothe end of the stereoscopic_video_format_descriptor( ).

A service_type field corresponds to an 8-bit field, which indicateswhether the video ES indicates by the stream_type field corresponds tothe video ES of a 2D image or the video ES of a 3D image. According tothe embodiment of the present invention, if the video ES corresponds tothe video ES of a 2D image, the service_type field value is marked as‘0’. Alternatively, if the video ES corresponds to the video ES of a 3Dimage, the service_type field value is marked as ‘1’.

A composition_type field corresponds to an 8-bit field, which indicatesby which transmission format the corresponding 3D content has beenreceived.

Herein, the composition_type field indicates by which of thetransmission formats, i.e., the side-by-side format, the top/bottomformat, the interlaced format, the frame sequential format, the checkerboard format, the anaglyph format, the full left/right format, the fullleft/half right format, and the 2D video/depth format, the corresponding3D image has been received. For example, when the composition_type fieldvalue is equal to ‘0x01’, the receiving system determines that thecorresponding 3D image has been received in a side-by-side format.

An LR_first_flag field indicates, when generating a stereo image (orwhen multiplexing a stereo image), whether the uppermost pixel of thefurthermost left side of the frame belongs to the left image, or whetherthe uppermost pixel of the furthermost left side of the frame belongs tothe right image. More specifically, the LR_first_flag field indicateswhether to display the furthermost left side of the received frame asthe left image, or whether to display the furthermost left side of thereceived frame as the right image. According to an embodiment of thepresent invention, if the value of the LR_first_flag field is equal to‘0’, the furthermost left side of the frame is displayed as the leftimage. And, if the value of the LR_first_flag field is equal to ‘1’, thefurthermost left side of the frame is displayed as the right image.

For example, when the transmission format is a side-by-side format, andif the value of the LR_first_flag field is equal to ‘0’, the receivingsystem decodes the pixels of the left-side half of a frame and displaysthe decoded pixels as the left image. And, the receiving system decodesthe pixels of the right-side half of the frame and displays the decodedpixels as the right image. Conversely, when the transmission format is aside-by-side format, and if the value of the LR_first_flag field isequal to ‘1’, the receiving system decodes the pixels of the left-sidehalf of a frame and displays the decoded pixels as the right image. And,the receiving system decodes the pixels of the right-side half of theframe and displays the decoded pixels as the left image.

As another example, when the transmission format is a top/bottom format,and if the value of the LR_first_flag field is equal to ‘0’, thereceiving system decodes the pixels of the upper half of a frame anddisplays the decoded pixels as the left image. And, the receiving systemdecodes the pixels of the lower half of the frame and displays thedecoded pixels as the right image. Conversely, when the transmissionformat is a top/bottom format, and if the value of the LR_first_flagfield is equal to ‘1’, the receiving system decodes the pixels of theupper half of a frame and displays the decoded pixels as the rightimage. And, the receiving system decodes the pixels of the lower half ofthe frame and displays the decoded pixels as the left image.

A spatial_flipping_flag field indicates whether at least one of the leftimage and the right image is inversely scanned and encoded. When thetransmitting system encodes a stereo image consisting of a left imageand a right image, the transmitting system scans the image by inversing(or flipping) the scanning direction of at least one of the left andright images, so as to enhance the coding efficiency. More specifically,depending upon the scanning efficiency, inverse scanning (or alignment)may be performed on the left or right image in a vertical or horizontaldirection. The inversely-scanned images will hereinafter be referred toas mirrored images for simplicity.

According to an embodiment of the present invention, when thetransmission format is a side-by-side format, the present inventionperforms inverse scanning on the left or right image in a horizontaldirection, thereby encoding the inversely-scanned image. And, when thetransmission format is a top/bottom format, the present inventionperforms inverse scanning on the left or right image in a verticaldirection, thereby encoding the inversely-scanned image. According tothe embodiment of the present invention, in this case, thespatial_flipping_flag field is marked to have the value of ‘1’. If thespatial_flipping_flag field value is equal to ‘1’, prior to displayingthe mirrored images, the received system inversely aligns the mirroredimages in the initial (or original) scanning order, thereby displayingthe aligned images. On the other hand, when the spatial_flipping_flagfield value is equal to ‘0’, this indicates that the pixels of the leftand right image are aligned in the initial scanning order, thereby beingencoded.

When the spatial_flipping_flag field value is equal to ‘1’, animage0_flipped_flag field indicates which image has been flipped (ormirrored or inverted). According to the embodiment of the presentinvention, if image( ) is flipped, then the image0_flipped_flag fieldvalue is equal to ‘1’. And, if imagel is flipped, theimage0_flipped_flag field is equal to ‘0’. Herein, image0 corresponds toan image having the uppermost pixel of the furthermost left side of aframe, which consists of left and right images, belonging thereto. And,image1 corresponds to the other image. More specifically, the mappingrelation between image0 and imagel and the left or right image is setbased upon the LR_first_flag field. If the LR_first_flag field is equalto ‘0’, the left image corresponds to image0, and the right imagecorresponds to image1.

A quincunx_filtering_flag field indicates whether a quincunx filter hasbeen used to perform sampling, when generating the stereo image.

According to an embodiment of the present invention, when thetransmitting system samples a left image or a right image to ahalf-resolution image, and if the quincunx filter has been used for thesampling process, the quincunx_filtering_flag field is marked to havethe value of ‘1’. Otherwise, the quincunx_filtering_flag field is markedto have the value of ‘0’. Herein, if the quincunx_filtering_flag fieldis equal to ‘1’, the receiving system performs an inverse process ofquincunx filtering on the corresponding image.

For example, in case of the side-by-side format, the top/bottom format,and the full left/half right format, when ½-sub-sampling the left orright image in a horizontal or vertical direction, and if the quincunxfilter has been used, the quincunx_filtering_flag field is marked tohave the value of ‘1’.

According to another embodiment of the present invention, in case of theside-by-side format, the top/bottom format, and the full left/half rightformat, when ½-sub-sampling the left or right image in a horizontal orvertical direction, a filter other than the quincunx filter may be used.For this case, the present invention may further include a fieldindicating the type of filter.

As described above, when the receiving system supports a transmissionformat indicated by the composition_type field value within thestereoscopic video format descriptorstereoscopic_video_format_descriptor( ), the system refers to otherfields within the stereoscopic_video_format_descriptor( ) so as todecode the corresponding 3D image and to display the decoded image.

The order, position, and definition of the fields allocated to thestereoscopic video format descriptorstereoscopic_video_format_descriptor( ), shown in FIG. 4, are merelyexamples presented to facilitate and simplify the understanding of thepresent invention. In other words, the order, position, and definitionof the fields allocated to the stereoscopic video format descriptorstereoscopic_video_format_descriptor( ) and the number of fields thatcan be additionally allocated thereto may be easily altered or modifiedby the system designer. Therefore, the present invention will not belimited to the examples given in the above-described embodiment of thepresent invention.

Herein, the additional 3D image information according to the presentinvention, i.e., the stereoscopic video format descriptor of FIG. 4 maybe included in a virtual channel table (VCT) and received.

FIG. 5 illustrates a block view showing the structure of a 3D imagingsystem according to the present invention. Herein, the 3D imaging systemincludes a content source 100, a decoding device 200, and a displaydevice 300. In the description of the present invention, the decodingdevice 200 and the display device 300 will be collectively referred toas a receiving system, for simplicity.

More specifically, the content source 100 includes 3D contents for the3D image. Examples of the content source 100 may include a disk, aserver, a terrestrial /satellite/cable broadcasting station.

The decoding device 200 receives content from the content source 100 anddecodes the received content, thereby creating an image suitable fordisplay. For example, if the received content is compression-encoded,the decoding device 200 performs decompression and/or interpolation,thereby recovering the received content (or image) back to its initialstate prior to being compression-encoded. Examples of the decodingdevice 200 may include a DVD player, a settop box, digital TV, and soon.

The display device 300 displays the image created in the decoding device200 either in a 2-dimensional (2D) format or in a 3-dimensional (3D)format. Examples of the display device 300 may include a screen, amonitor, a projector, and so on.

Furthermore, the display device 300 may also correspond to a device thatcan display general 2D images, a device that can display 3D imagesrequiring special viewing glasses, a device that can display 3D imageswithout requiring any special viewing glasses, and so on.

More specifically, by using at least two images based upon thecharacteristics of the display device 300, the receiving system createsand displays a 3D image using a variety of methods. For example, thedisplay method may include a method of wearing special glasses, and amethod of not wearing any glasses.

The method of wearing special glasses is then divided intro a passivemethod and an active method. The passive method corresponds to a methodof showing the 3D image by differentiating the left image and the rightimage using a polarizing filter. More specifically, the passive methodcorresponds to a method of wearing a pair of glasses with one red lensand one blue lens fitted to each eye, respectively. The active methodcorresponds to a method of differentiating the left image and the rightimage by sequentially covering the left eye and the right eye at apredetermined time interval. More specifically, the active methodcorresponds to a method of periodically repeating a time-split (ortime-divided) and viewing the corresponding image through a pair ofglasses equipped with electronic shutters which are synchronized withthe time-split cycle period of the image. The active method may also bereferred to as a time-split method or a shuttered glass method.

The most well-known methods of not wearing any glasses include alenticular method and a parallax barrier method. Herein, the lenticularmethod corresponds to a method of fixing a lenticular lens panel infront of an image panel, wherein the lenticular lens panel is configuredof a cylindrical lens array being vertically aligned. The parallaxmethod corresponds to a method of providing a barrier layer havingperiodic slits above the image panel.

At this point, the decoding device 200 and the display device 300 of thereceiving system may be implemented as separate bodies or may beincorporated into a single body.

According to an embodiment of the present invention, in case thedecoding device 200 and the display device 300 of the receiving systemare implemented as separate bodies, each of the decoding device 200 andthe display device 300 uses a digital interface in order to transmitand/or receive data.

Examples of the digital interface may include a digital visual interface(DVI), a high definition multimedia interface (HDMI), and so on. Thedescription of the present invention introduces a method of using theHDMI as the digital interface according to the embodiment of the presentinvention. In order to do so, the decoding device 200 and the displaydevice 300 are interconnected through an HDMI cable. The HDMI transmitsdigital video signals and digital audio signals at a bandwidth equal toor greater than 5 Gbps.

FIG. 6 illustrates a block view showing an exemplary structure of thedecoding device 200 and the display device 300 of the receiving systemaccording to the present invention being interconnected through an HDMIcable 400.

When the decoding device 200 and the display device 300 areinterconnected through the HDMI cable 400, the decoding device 200 willbe referred to as an HDMI source, and the display device 300 will bereferred to as an HDMI sink. According to the embodiment of the presentinvention, the HDMI source corresponds to a settop box. Additionally, inthe description of the present invention, the decoding device 200 mayalso be referred to as an image receiving unit, and the display device300 may also be referred to as a display unit.

Referring to FIG. 6, the HDMI source 200 includes a demultiplexer 201, avideo decoder 202, a data decoder 203, and an HDMI transmitter 204.

The HDMI sink 300 includes an HDMI receiving unit 301, a 3D formatter302, and a display engine 303.

In the present invention, it is assumed that an MPEG-2 transport stream(TS) packet received from the content source 100 and being demodulatedis inputted to the demultiplexer 201. Herein, the demodulated TS packetmay correspond to a TS packet of a 2D image or a TS packet of a 3Dimage.

The demultiplexer 201 receives the TS packet so as to performdemultiplexing. The TS packet comprises a header and a payload. Herein,the header includes a PID, and the payload includes one of a videostream, an audio stream, and a data stream. The demultiplexer 201 usesthe PID of the TS Packet being inputted so as to separate the videostream, the audio stream, and the data stream from the corresponding TSpacket. The separated video stream is outputted to the video decoder202, and the data stream including the system information is outputtedto the data decoder 203. The separated audio stream is outputted to theaudio decoder. However, since the audio decoder does not correspond toone of the characteristics of the present invention, detaileddescription of the audio decoder will be omitted for simplicity.

The video decoder 202 performs decoding on the video stream based upon apre-determined video decoding algorithm, thereby recovering the receivedvideo stream to the initial video stream prior to compression. Examplesof the video decoding algorithm includes an MPEG-2 video decodingalgorithm, an MPEG-4 video decoding algorithm, an H.264 decodingalgorithm, an SVC decoding algorithm, a VC-1 decoding algorithm, and soon. Since it is assumed in the present invention that the video streamis MPEG-2 compression-encoded, the video decoder 202 uses the MPEG-2video decoding algorithm.

The video stream decoded by the video decoder 202 is then outputted tothe HDMI transmitting unit 204.

The data decoder 203 uses a table_id and a section_length of the systeminformation so as to identify various tables. Then, the data decoder 203parses sections of the identified tables. Thereafter, the data decoder203 either stores the parsed result in a storage device as database oroutputs the parsed result to the HDMI transmitting unit 204. Forexample, the data decoder 203 may group (or collect) sections having thesame table identifier (table_id), so as to configure a table.

Furthermore, among the tables identified from the system information,the data decoder 203 parses the stereoscopic video format descriptorfrom the PMT, thereby outputting the parsed result to the HDMItransmitting unit 204.

The HDMI transmitting unit 204 receives the decoded video stream andperforms transition minimized differential signaling interface(TMDS)-encoding on the received decoded video stream. Subsequently, theHDMI transmitting unit 204 outputs the TDMS-encoded video stream to theHDMI receiving unit 301 of the HDMI sink 300 through the HDMI cable 400.

Moreover, the HDMI transmitting unit 204 uses the additional 3D imageinformation acquired from the stereoscopic video format descriptor so asto generate (or convert) 3D signaling information (or 3D signalingdata). Thereafter, the HDMI transmitting unit 204 outputs the generated(or converted) 3D signaling information (or 3D signaling data) to theHDMI receiving unit 301 of the HDMI sink 300 through the HDMI cable 400.More specifically, the HDMI transmitting unit 204 of the HDMI source 200uses the TDMS channel to output TDMS-encoded video data and 3D signalinginformation to the HDMI receiving unit 301 of the HDMI sink 300.

Herein, the TDMS channel is used for transmitting video, audio, andsupplementary data. At this point, the HDMI transmitting unit 204 uses apacket structure in order to transmit the supplementary data.

FIG. 7 illustrates a table showing examples of diverse packet types usedin an HDMI standard according to the present invention.

Referring to FIG. 7, when the packet type value is equal to ‘0x82’, thisindicates that the packet structure corresponds to an auxiliary videoinformation (AVI) InfoFrame.

According to an embodiment of the present invention, the 3D signalinginformation is included in the AVI InfoFrame so as to be transmitted tothe HDMI sink 300.

More specifically, in order to receive and/or transmit data to and fromthe HDMI source 200 and the HDMI sink 300, the link of the HDMI may bebroadly divided into a video data period, a data island period, and acontrol period.

During the video data period, active pixels of an active video line aretransmitted. And, during the data island period, audio and supplementaldata are transmitted through a series of packets. The control period isused when the video, audio, and supplemental data are not required to betransmitted.

According to the embodiment of the present invention, the HDMItransmitting unit 204 of the HDMI source 200 outputs an AVI InfoFramepacket including the 3D signaling information to the HDMI receiving unit301 of the HDMI sink 300 based upon the data island period.

Herein, the AVI InfoFrame packet configured of a header region and acontents region.

FIG. 8 illustrates an exemplary header structure of an AVI InfoFramepacket according to the embodiment of the present invention. In theexample shown in FIG. 8, the header structure of the AVI InfoFramepacket is configured of 3 bytes. Herein, the 1^(st) byte (HB0) indicatesthe packet type, the 2^(nd) byte (HB1) indicates the versioninformation, and the lower 5 bits of the 3^(rd) byte (HB2) indicate thecontents length of the AVI InfoFrame packet in byte units.

The packet type value ‘0x82’ is indicated in the 1^(st) byte (HB0)configuring the header of the AVI InfoFrame packet according to thepresent invention.

FIG. 9 illustrates a contents structure of a general AVI InfoFramepacket according to the embodiment of the present invention. Herein, the3^(rd) byte (HB2) within the header of the AVI InfoFrame packet ismarked to have a contents length value of ‘0x0D’ (i.e., ‘13’ in decimalnumbers).

Additional video information are included in the 1^(st) byte (PB0) tothe 14^(th) byte (PB13) of the AVI InfoFrame packet contents of FIG. 9.For example, bar information is recorded in the 7^(th) byte (PB6) to the14^(th) byte (PB13).

In the AVI InfoFrame packet contents of FIG. 9, the region starting fromthe 15^(th) byte (PB14) to the 28^(th) byte (PM27) corresponds to anunused region.

According to the embodiment of the present invention, the presentinvention uses one of the unused bytes of the AVI InfoFrame packetcontents shown in FIG. 9, so as to record the 3D signaling information.

According to the embodiment of the present invention, the presentinvention uses the 15^(th) byte (PB 14) to record the 3D signalinginformation.

At this point, the contents length value in the AVI InfoFrame packetheader of FIG. 8 is modified to ‘0x0E’ (i.e., ‘14’ in decimal numbers).

FIG. 10 illustrates a content structure of an AVI InfoFrame packetaccording to the embodiment of the present invention.

Referring to FIG. 10, additional video information are marked from the1^(st) byte (PB0) to the 14^(th) byte (PB13) of the AVI InfoFrame packetcontents, and 3D signaling information according to the presentinvention is marked (or recorded) in the 15^(th) byte (PB 14).Furthermore, the region starting from the 16^(th) byte (PB15) to the28^(th) byte (PM27) corresponds to an unused region reserved for futureusage.

Herein, according to the embodiment of the present invention, the 3Dsignaling information being marked in the 15^(th) byte (PB 14) isgenerated (or created) based upon the additional 3D image informationacquired from the stereoscopic video format descriptor included andreceived in the PMT.

The HDMI transmitting unit 204 allocates (or assigns) a 1-bit SV field,a 3-bit CT field (CT2 to CT0), a 1-bit OR field, a 2-bit FL field (FL1and FL0), and a 1-bit QS field to the 15^(th) byte (PB 14) region, so asto mark (or indicate) the 3D signaling information.

More specifically, the 3D signaling information is generated by usingthe additional 3D image information acquired from a service_type field,a composition_type field, an LR_first_flag field, aspatial_flipping_flag field, an image0_flipped_flag field, and aquincunx_filtering_flag field of the stereoscopic video formatdescriptor.

For example, the HDMI transmitting unit 204 indicates (or marks) theinformation obtained from the service_type field in the SV field,indicates the information obtained from the composition_type field inthe CT field, and indicates the information obtained from theLR_first_flag field in the OR field. Also, the HDMI transmitting unit204 indicates the information obtained from the spatial_flipping_flagfield and the image0_flipped_flag field in the FL field, and indicatesthe information obtained from the quincunx_filtering_flag field in theQS field.

(a) to (e) of FIG. 11 respectively illustrate a set of tables indicatingthe meaning (or definition) of the values assigned to each field withinthe 15^(th) byte (PB14) of the AVI InfoFrame packet according to theembodiment of the present invention.

More specifically, (a) of FIG. 11 illustrates an example of the SVfield. Herein, according to the embodiment of the present invention,when the service_type field value indicates a 2D image, the SV fieldindicates ‘0’. And, when the service_type field value indicates a 3Dimage, the SV field indicates ‘1’.

(b) of FIG. 11 illustrates an example of the CT field. Herein, when theSV field indicates a 3D image, the transmission format acquired from thecomposition_type field is indicated in the CT field. For example, whenthe transmission format corresponds to the top/bottom format, the CTfield indicates ‘000’. Alternatively, when the transmission formatcorresponds to the side-by-side format, the CT field indicates ‘001’.

(c) of FIG. 11 illustrates an example of the OR field. When theLR_first_flag field indicates that the uppermost pixel of the left-endportion within the frame corresponds to the left image, the OR fieldindicates ‘0’. And, when the LR_first_flag field indicates that theuppermost pixel of the left-end portion within the frame corresponds tothe right image, the OR field indicates ‘1’.

(d) of FIG. 11 illustrates an example of the FL field. Herein, the FLfield may be determined by using the spatial_flipping_flag field valueand the image0_flipped_flag field value.

For example, when the spatial_flipping_flag field value is ‘0’, i.e.,when it is indicated that the right image and the left image are bothscanned and encoded in their original orders, the upper bit (FL1) of theFL field is marked as ‘1’. Conversely, when the spatial_flipping_flagfield value is ‘1’, i.e., when it is indicated that at least one of theright image and the left image is inversely scanned and encoded, theupper bit (FL1) of the FL field is marked as ‘0’. At this point, whenimage1 is flipped, the lower bit (FL0) of the FL field is marked as ‘0’.Alternatively, when image( ) is flipped, the lower bit (FL0) of the FLfield is marked as ‘1’.

(e) of FIG. 11 illustrates an example of the QS field. According to theembodiment of the present invention, when the quincunx_filtering_flagfield value does not indicate the usage of the filter, the QS fieldindicates ‘0’. And, when the quincunx_filtering_flag field valueindicates the usage of the filter, the QS field indicates ‘1’.

As described above, the AVI InfoFrame packet having the 3D signalinginformation recorded therein is transmitted to the HDMI receiving unit301 of the HDMI sink 300 through the HDMI cable 400 along with thedecoded video data.

The HDMI receiving unit 301 outputs the received video data to the 3Dformatter 302 and parses each field of the 15^(th) byte (PB 14) withinthe AVI InfoFrame packet contents, thereby acquiring the 3D signalinginformation. The acquired 3D signaling information is then outputted tothe 3D formatter 302 and the display engine 303. More specifically, wheneach field of the 15^(th) byte (PB14) within the AVI InfoFrame packetcontents is parsed, the HDMI receiving unit 301 may be able to determine(or know) whether the received video data correspond to a 2D image or toa 3D image. Also, if the received video data correspond to a 3D image,the HDMI receiving unit 301 may be able to determine (or know) thetransmission format. Furthermore, the HDMI receiving unit 301 may alsobe able to determine (or know) whether the uppermost pixel of theleft-end portion within the frame of the received 3D image belongs tothe left image or to the right image, whether at least one of the leftimage and the right image has been inversely scanned and encoded, andwhich of the left image and the right image has been inversely scanned.Finally, the HDMI receiving unit 301 may also be able to determine (orknow) whether at least one of the left image and the right image hasbeen sampled by using a filter.

The 3D formatter 302 refers to the 3D signaling information outputtedfrom the HDMI receiving unit 301, in order to re-format the video dataoutputted from the HDMI receiving unit 301, so that the re-formattedvideo data can fit the display format of the HDMI sink 300, therebydisplaying the re-formatted video data to the display engine 303. Thedisplay engine 303 displays the 3D image outputted from the 3D formatter302 in accordance with its display method. More specifically, thedisplay engine 303 creates a 3D image through diverse methods using aleft image and a right image based upon the display characteristics ofthe HDMI sink, thereby displaying the created 3D image. As describedabove, the display methods include a method of wearing special viewingglasses and a method of not wearing any special viewing glasses.

For example, it is assumed that the SV field value acquired from the15^(th) byte (PB14) within the AVI InfoFrame packet contents indicatesthat the corresponding video data is a 3D image, that the CT field valueindicates that the transmission format is a side-by-side format, thatthe OR field value indicates ‘0’, that the FL field value indicates‘11’, and that the QS field value indicates ‘1’. In this case, it can bedetermined that the uppermost pixel of the left-end portion within thereceived video data frame belongs to the left image, that the rightimage has been inversely scanned during the encoding process, and that aquincunx filter has been used when sampling the left image and the rightimage. Accordingly, among the video data, the 3D formatter 302 scans theright image in an inverse direction and decodes the inversely scannedright image. At this point, by performing an inverse process of thequincunx filter or an adequate inverse-filtering process, thecorresponding image may be recovered to its original (or initial) size.The display engine 303 displays the image having the left-half of thepixels within a single frame decoded and being recovered, as the leftimage. Also, the display engine 303 displays the image having theright-half of the pixels within the frame decoded and being recovered,as the right image.

FIG. 12 illustrates a flow chart showing process steps for generating 3Dsignaling information by acquiring additional 3D image information fromthe PMT from the HDMI source 200 and transmitting the generated 3Dsignaling information through the HDMI cable according to the embodimentof the present invention.

More specifically, the HDMI source 200 finds (or searches) a PAT havingPID=0 from the inputted data stream (S401). Thereafter, the PID of thePMT is acquired from the PAT, and stream packets having the acquired PIDof the PMT are grouped (or collected) so as to configure the PMT (S402).Subsequently, the additional 3D image information is acquired from thestereoscopic video format descriptor of the PMT (S403). In other words,the additional 3D image information is acquired from the service_typefield, the composition_type field, the LR_first_flag field, thespatial_flipping_flag field, the image0_flipped_flag field, and thequincunx_filtering_flag field of the stereoscopic video formatdescriptor.

Furthermore, 3D signaling information is generated by using the acquiredadditional 3D image information. Then, after recording the generated 3Dsignaling information in the 15^(th) byte (PB 14) of the AVI InfoFramepacket contents, the 3D signaling information is outputted to the HDMIsink 300 through the HDMI cable 400 (S404). According to the embodimentof the present invention, the information obtained (or acquired) fromthe service_type field is marked (or indicated) in the SV field of the15^(th) byte (PB14) of the AVI InfoFrame packet contents. Theinformation obtained from the composition_type field is indicated in theCT field. And, the information obtained from the LR_first_flag field ismarked in the OR field. Also, the information obtained from thespatial_flipping_flag field and the image0_flipped_flag field isindicated in the FL field, and the information obtained from thequincunx_filtering_flag field is marked in the QS field.

FIG. 13 illustrates a flow chart showing process steps of receivingvideo data and 3D signaling information from the HDMI sink 300 throughthe HDMI cable and displaying the received video data and 3D signalinginformation for processing video data according to the embodiment of thepresent invention. More specifically, the video data and the AVIInfoFrame packet are received through the HDMI cable 400 (S501). Then,3D signaling information is obtained from the 15^(th) byte (PB 14) ofthe AVI InfoFrame packet. The video data are re-formatted to fit thedisplay format of the HDMI sink 300 based upon the obtained 3D signalinginformation (S502). Thereafter, a 3D image is created through diversemethods by using the left image and the right image of the re-formattedvideo data, thereby being displayed (S503).

As described above, when a decoding device and a display device areinterconnected through an HDMI cable, additional 3D image information isobtained from a PMT being received by the decoding device, so as tocreate 3D signaling information from the obtained additional 3D imageinformation. Subsequently, the created 3D signaling information isinserted in the AVI InfoFrame packet, thereby outputted to the displaydevice through the HDMI cable. Thus, the display device may be able toaccurately recover the 3D image and display the recovered 3D image.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the inventions. Thus, itis intended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

What is claimed is:
 1. A method of processing a 3-dimensional, 3D,broadcast service in a digital broadcast receiver, the methodcomprising: receiving a broadcast signal including 3D broadcast servicedata and service information for signaling the 3D broadcast servicedata, wherein the service information includes first informationindicating that a transmission format of the 3D broadcast servicecorrespond to a line alternative format in which pixels of a left imageand pixels of a right image are alternated line by line; decoding the 3Dbroadcast service data and the service information; transition MinimizedDifferential Signaling, TMDS, coding the decoded 3D broadcast servicedata and generating an InfoFrame packet for signaling the TMDS coded 3Dbroadcast service data using the service information; transmitting theTMDS coded 3D broadcast service data and the generated InfoFrame packetto a display unit, wherein the InfoFrame packet includes a first fieldhaving information derived from the first information.
 2. The method ofclaim 1, wherein the service information includes second informationindicating a quincunx filtering is applied to at least one of the leftimage and right image, wherein the InfoFrame packet further includes asecond field having information derived from the second information. 3.The method of claim 2, wherein the generated InfoFrame packetcorresponds to an Auxiliary Video Information, AVI, InfoFrame packet. 4.The method of claim 3, wherein the AVI InfoFrame packet comprises aheader and a contents region, and wherein one or more fields areassigned to at least one byte of the contents region, thereby recordingsignaling information for the 3D broadcast service data.
 5. An apparatusfor receiving a 3 dimensional, 3D, broadcast service, the apparatuscomprising: receiving unit configured to receive a broadcast signalincluding 3D broadcast service data and service information forsignaling the 3D broadcast service data, wherein the service informationincludes first information indicating that a transmission format of the3D broadcast service correspond to a line alternative format in whichpixels of a left image and pixels of a right image are alternated lineby line; decoding unit configured to decode the 3D broadcast servicedata and the service information; conversion unit configured toTransition Minimized Differential Signaling, TMDS, code the decoded 3Dbroadcast service data and generate an InfoFrame packet for signalingthe TMDS coded 3D broadcast service data using the service information;transmission unit configured to transmit the TMDS coded 3D broadcastservice data and the generated InfoFrame packet to a display unit,wherein the InfoFrame packet includes a first field having informationderived from the first information.
 6. The apparatus of claim 5, whereinthe service information includes second information indicating aquincunx filtering is applied to at least one of the left image andright image, wherein the InfoFrame packet further includes a seoncdfield having information derived from the second information.
 7. Theapparatus of claim 6, wherein the generated InfoFrame packet correspondsto an Auxiliary Video Information, AVI, InfoFrame packet.
 8. Theapparatus of claim 7, wherein the AVI InfoFrame packet comprises aheader and a contents region, and wherein one or more fields areassigned to at least one byte of the contents region, thereby recordingsignaling information for the 3D broadcast service data.